Processing Information Graphics in Multimodal Documents

Information graphics, such as bar charts, grouped bar charts, and line graphs, are an important component of multimodal documents and cannot be ignored. When such graphics appear in popular media, such as magazines and newspapers, they generally have an intended message. We argue that this message represents a brief summary of the graphic’s high-level content, and thus can serve as the basis for more robust information extraction from multimodal documents. The paper describes our methodology for automatically recognizing the intended message of an information graphic, with a focus on grouped bar charts.